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FINAL  REPORT 

SINGULAR  AND  BANG-BANG  STOCHASTIC  CONTROL 
AFOSR  GRANT  NUMBER  85-0360 
October  1,  1985  -  September  30,  1988 


Steven  E.  Shreve,  Principal  Investigator  Department  of  Mathematics 

Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

The  proposal  which  formed  the  basis  for  this  grant  consisted  of  three 
parts:  research  on  singular  stochastic  control,  research  on  bang-bang 
stochastic  control,  and  the  creation  of  a  graduate  text  which  would 
make  these  two  topics  more  widely  accessible.  Significant  progress 
has  been  made  in  the  two  research  areas,  although  a  number  of 
important  questions  remain.  The  proposed  graduate  text  has  been 
completed  and  a  copy  was  provided  to  the  Air  Force  Office  of 
Scientific  Research  in  December  1987.  We  discuss  each  of  these 
three  topics  in  turn. 

1.  Singular  Stochastic  Control.  Singular  stochastic  control  was 
designed  to  model  the  limiting  behavior  of  a  control  system  in  which 
instantaneous  displacements  in  the  state  can  be  caused  by  the 
control.  When  these  displacements  are  small  and  frequent,  the 
model  of  singularly  continuous  displacement  is  appropriate.  Such 
problems  were  originally  motivated  by  thruster  control  of  spacecraft, 
but  they  have  more  recently  been  used  in  the  study  of  queueing 
systems. 


In  our  original  proposal  we  identified  two  important  problem  areas 
in  singular  stochastic  control.  The  first  was  the  area  of  finite-fuel 
problems,  i.e.,  control  of  systems  in  which  the  maximum  amount  of 
displacement  the  controller  is  allowed  to  cause  is  some  finite  number 
known  beforehand.  Such  problems  are  in  the  spirit  of  the  original 
application  in  control  of  spacecraft.  Our  proposal  was  to  adapt 
techniques  we  had  used  in  the  analysis  of  one-dimensional  problems, 
including  their  reduction  to  problems  of  optimal  stopping,  to  study 
finite-fuel  problems  in  which  the  amount  of  fuel  remaining 
constituted  a  second  state  variable.  This  approach  was  shown  to  be 
viable  in  Karatzas  &  Shreve  [1].  Work  in  this  direction  is  continuing 
with  the  assistance  of  graduate  students  who  have  been  partially 
supported  by  the  grant. 
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The  second  area  of  singular  stochastic  control  in  which  we  proposed 
to  focus  our  efforts  involved  generalization  of  known  results  to 
higher  dimensions.  One  issue  we  wished  to  address  in  this  regard 
was  the  regularity  of  the  value  function  in  higher  dimensions.  The 
continuity  of  the  second  derivative  of  this  function  played  an 
important  part  in  the  solution  of  a  number  of  one-dimensional 
problems,  but  there  was  no  theory  suggesting  that  this  high  degree 
of  smoothness  would  be  present  in  higher  dimensional  problems.  It 
has  now  been  determined  in  joint  work  with  Mete  Soner  [2]  that  the 
hoped  for  smoothness  is  present  in  fairly  general  two-dimensional 
problems.  Our  continuing  efforts  are  directed  toward  establishing  a 
similar  result  in  more  than  two  dimensions. 

Another  facet  of  singular  control  was  studied  by  Arthur  Heinricher 
and  Victor  Mizel  while  the  former  was  partially  supported  by  this 
grant.  A  deterministic  control  problem  which  exhibited  the 
Lavrentiev  phenomenon  was  rendered  stochastic  by  the  addition  of 
white  noise  to  the  state  equation.  The  Lavrentiev  phenomenon 
occurs  when  the  minimum  cost  attainable  by  using  control  laws 
which  are  Lipschitz  continuous  in  time  is  strictly  larger  than  the  cost 
attainable  by  using  some  larger  class  of  control  laws.  This  has 
important  ramifications  which  one  considers  numerical 
approximation  schemes.  When  the  stochastic  version  of  a  particular 
Lavrentiev  deterministic  problem  was  examined,  it  was  discovered 
that  the  minimum  expected  cost  attainable  by  using  control 
processes  which  were  absolutely  continuous  functions  of  time  was 
strictly  larger  than  the  minimum  attainable  using  singularly 
continuous  processes.  Approximation  of  singularly  continuous 
stochastic  control  problems  by  absolutely  continuous  ones  has  been 
used  as  both  a  theoretical  and  a  numerical  tool,  and  the  Heinricher- 
Mizel  example  indicates  the  limitations  of  that  approach.  Their  work 
is  reported  in  [3,4]. 

2.  Bang-Bang  Stochastic  Control.  Bang-bang  stochastic  control 
problems  are  those  in  which  (he  drift  of  the  controlled  process 
changes  discontinuously  as  the  state  crosses  certain  boundaries. 

Using  the  Girsanov  change  of  measure  theorem  and  Brownian  local 
time,  it  is  possible  in  some  cases  to  compute  explicitly  the  transition 
densities  for  processes  controlled  in  this  way.  In  work  supported  by 
a  previous  proposal,  loannis  Karatzas  and  I  noted  a  remarkable 
coincidence  involving  the  joint  distributions  of  Brownian  motion,  its 
local  time,  occupation  time,  the  time  it  attained  its  maximum,  and  the 
value  of  this  maximum.  We  sought  an  explanation  of  this  coincidence 


by  decomposing  the  Brownian  path  and  reconstructing  it  so  that 
some  of  these  objects  mapping  into  other  of  these  objects.  That 
project  is  now  completed  and  is  reported  in  [5].  The  methodology  of 
path  decomposition  applied  here  is  currently  receiving  much 
attention  by  probabilists,  and  it  may  be  that  there  are  additional 
applications  for  control  theory. 

3.  Brownian  Motion  and  its  Local  Time.  The  theory  of  lio 
processes  and  the  application  of  this  theory  to  stochastic  control  has 
reached  a  level  of  development  which  justifies  a  textbook  treatment 
of  these  topics.  In  particular,  we  found  the  difficult  and  esoteric 
concept  of  Brownian  local  time  to  be  an  indispensible  tool  in  dealing 
with  a  number  of  stochastic  control  problems,  but  we  were  unable  to 
find  a  textbook  we  could  recommend  to  our  graduate  students  for  a 
treatment  of  this  and  related  topics.  For  this  reason,  loannis  Karatzas 
and  I  undertook  to  write  a  graduate  text  on  Brownian  motion, 
stochastic  integration  with  respect  to  continuous,  square-integrable 
martingales,  strong  and  weak  solutions  of  stochastic  differential 
equations,  and  Brownian  local  time.  The  project  actually  began  in  the 
fall  of  1984  and  was  concluded  with  the  appearance  of  the  book  in 
November  1987.  The  book  has  470  pages,  and  in  addition  to  the 
presentation  of  theory,  contains  about  275  problems  and  exercises, 
many  with  solutions.  It  is  our  goal  to  make  the  topics  treated  under 
this  grant  accessible  to  the  mathematics,  engineering  and  economics 
communities. 
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